User interface for a remote control application

ABSTRACT

A hand-held electronic device having a remote control application user interface that functions to displays operational mode information to a user. The graphical user interface may be used, for example, to setup the remote control application to control appliances for one or more users in one or more rooms, to perform activities, and to access favorites. The remote control application is also adapted to be upgradeable. Furthermore, the remote control application provides for the sharing of operational mode information.

RELATED APPLICATION INFORMATION

This application is a divisional of U.S. application Ser. No.11/340,442, filed Jan. 26, 2006, which application was filed as adivisional of U.S. application Ser. No. 10/288,727, filed Nov. 6, 2003,which, in turn, claims the benefit of U.S. Provisional PatentApplication Nos. 60/344,020 filed on Dec. 20, 2001 and 60/334,774 filedon Nov. 20, 2001.

These applications, from which priority is claimed, are herebyincorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

This invention relates generally to hand-held electronic devices and,more particularly, relates to a user interface for an universal remotecontrol application resident on the hand-held electronic device.

Universal remote controls for controlling the operation of multiple,diverse home appliances are well known. In this regard, universal remotecontrols perform a valuable function by consolidating three, four, five,and more remote controls into one device. However, as more remotelycontrollable appliances enter the homes of consumers and the number ofremotely controllable operations increase, the user interface of theuniversal remote control becomes increasingly more complex. Thiscomplexity arises from the need to provide more and more keys which areused to initiate the transmission of the control codes that control theincreasing number of operations of the increasing number of homeappliances. Disadvantageously, as the user interface of the universalremote control becomes more cluttered, the usability of the universalremote control diminishes. Accordingly, a need exists for a universalremote control having an improved user interface that simplifies theoperation of the universal remote control and, as such, the remoteoperation of consumer appliances.

SUMMARY OF THE INVENTION

In accordance with these needs, the subject invention is directed to ahand-held electronic device having a remote control application userinterface that functions to display operational mode information to auser. Representative platforms for the hand-held electronic deviceinclude, but are not limited to, devices such as personal digitalassistants, Web tablets, lap-top computers, extended-functionalitymobile phones, remote control devices, etc. which are collectivelyreferred to hereinafter as “PDAs.” An understanding of the objects,advantages, features, properties and relationships of the remote controlapplication user interface will be obtained from the following detaileddescription and accompanying drawings which set forth illustrativeembodiments which are indicative of the various ways in which theprinciples of the remote control application user interface may beemployed.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the remote control application userinterface, reference may be had to various preferred embodiments shownin the following drawings in which:

FIG. 1 illustrates an exemplary system in which a PDA is adapted tocontrol consumer appliances;

FIG. 2 illustrates an exemplary system in which a PDA receives updatesand supplemental IR codes from a Web site;

FIG. 3 illustrates the internal topology of an exemplary system in whicha PDA is adapted to control consumer appliances;

FIG. 4 illustrates an exemplary hardware circuit for enabling a PDA totransmit consumer appliance IR codes;

FIG. 5 illustrates an exemplary remote control application softwarearchitecture for a PDA;

FIG. 6 illustrates an exemplary hardware communication manager softwarearchitecture for a PDA;

FIG. 7 illustrates an exemplary file I/O manager software architecturefor a PDA;

FIGS. 8 and 9 illustrate exemplary PDA display screens illustrating aresident or installed remote control application;

FIG. 10 illustrates an exemplary PDA physical appearance and layout;

FIG. 11 illustrates an exemplary remote control application home screenwith the remote control application configured to control the operationof consumer appliances;

FIG. 12 illustrates an exemplary remote control application home screenprior to the remote control application being configured to control theoperation of consumer appliances;

FIG. 13 illustrates exemplary remote control application hard keyfunction assignments on a PDA together with corresponding informationaldisplays;

FIG. 14 illustrates an exemplary series of steps to set up the remotecontrol application to command the operation of a new consumerappliance;

FIG. 15 illustrates an exemplary series of steps to set up a new user ofthe remote control application;

FIG. 16 illustrates an exemplary series of steps to set up the remotecontrol application to command the operation of consumer appliances in adesignated room;

FIG. 17 illustrates an exemplary series of steps to set up a new remotecontrol application favorite channel soft key;

FIG. 18 illustrates an exemplary set of data relationships for executionof a remote control application macro function;

FIG. 19 illustrates exemplary remote control application device controlpages;

FIG. 20 illustrates an exemplary user customization of a remote controlapplication device control page;

FIG. 21 illustrates an exemplary automatic customization of a remotecontrol application device control page;

FIG. 22 illustrates an exemplary series of steps to program a remotecontrol application activity or macro;

FIG. 23 illustrates an exemplary automatic adjustment of a remotecontrol application device control page based on a geographical region;

FIG. 24 illustrates an exemplary start-up initialization sequence forthe remote control application.

DETAILED DESCRIPTION

Exemplary embodiments of a user interface for an universal remotecontrol application resident on the hand-held electronic device areillustrated and described which provide a next generation of homeautomation and consumer electronic controls. Representative platformsfor the hand-held electronic device include devices supporting theWindows CE brand operating system or PocketPC type devices that includebuilt-in support for transmitting consumer equipment IR remote controlcodes. Exemplary devices include, but are not limited to, devices suchas the Compaq (now HP) iPAQ brand model 3950 or 3970 that include ahardware module to house the IR transmitter and an IR database ofconsumer electronics command codes, along with a remote controlapplication that is used to control the operations of various consumerappliances. Other platform implementations using either built-in oradd-on hardware to implement the IR transmit functionality are alsopossible, using other Windows CE brand class devices or other softwaresystems such as, for example, the Palm brand operating system. While thedescriptions that follow are particularly illustrated using an iPAQPocketPC brand platform, it will be appreciated by those of ordinaryskill in the art that these teachings can be easily adapted to otherplatforms and software systems, for example, by making simpleadjustments to display graphic sizes and layouts, file formats, etc.,where needed to adapt to the different operating environment.

Turning to FIGS. 1 and 2 there is illustrated a system in which a PDAdevice 1150 is adapted to provide wireless control of consumerappliances such as home entertainment equipment 1120, lighting 1122, andtemperature control 1124. Control signals are provided, for example, bycausing the PDA 1150 to emulate the wireless signals (IR, RF, etc.) usedby the standard remote controls originally supplied with the respectiveappliances. The PDA 1150 is also preferably adapted to interact withWeb-hosted services 1106, such as IR code updates, graphics, softwareenhancements, electronic program guides (EPGs), etc., via a host desktopcomputer 1152 and docking station 1151 as particularly illustrated inFIG. 2. It will be appreciated by those skilled in the art that theWeb-hosted services may also be accessed directly, sans an interveninghost desktop computer, in cases where the PDA 1150 is adapted tocommunicate directly to the Internet. Still further, the PDA 1150 may beadapted to share all or part of a user's programmed configurationsettings with other compatible PDAs 1150′. An exemplary graphicalrepresentation of the topology of such an exemplary system isillustrated in FIG. 3 which depicts the major components of the remotecontrol application and the environment in which it operates.Represented in FIG. 3 are the PDA 1150, the host desktop PC 1152, andWeb Server(s) accessible through the Internet 1154.

The PDA Software Application 1100 described hereinafter and showngraphically in FIG. 3 generally comprises four program segments thatwould typically be installed on the PDA 1150 in order to control, forexample, home entertainment components 1120 and home automationcomponents such as lights 1122 and climate control 1124. By way ofillustrative example, the four program segments comprise:

Remote Control Software 1102: This application allows the user tocontrol their consumer electronic devices. The application providesusers with an easy and intuitive way to manage their home entertainmentequipment as well as home automation devices (such as X10 type devices)within the home. The remote control application itself may comprise fourmajor architectural subcomponents, although only the top level (referredto as the “presentation engine”) is exposed to the end user. These fourcomponents, the presentation engine 1108, the configuration engine 1110,the hardware communication manager 1114, and the file I/O manager 1112will be described in more detail hereinafter.

ActiveSync Software 1116: This application is used as a communicationsconduit to and from a remote control companion application that isinstalled on the host desktop PC 1152.

IR Signaling Software 1118: This application performs the IR signalgenerating functions to universally control appliances; supporting adatabase of IR codes and transmission protocols for use in conjunctionwith the IR hardware components described later with reference to FIG.4.

RF Module Software/Hardware 1119: A Bluetooth or 802.11 RF module mayoptionally perform the RF signal generation to control RF devices and/orcommunicate wirelessly with a server or other applications resident onthe host desktop PC 1152. It will be appreciated that such an RF Modulemay be used either together with or in place of the above mentioned IRSignaling to effect control of home appliances.

As noted, the ActiveSync software 1116 cooperates with a remote controlcompanion application 1104 installed on the host desktop PC 1152. Thiscombination of applications generally implements a standardMicrosoft-based ActiveSync brand utility that functions as acommunications conduit to/from the remote control application. Softwaresupplemental to the ActiveSync brand applications may be required toensure that data required for the remote control application is passedalong the conduit in an appropriate fashion. Such supplemental softwarewould be used to restore/backup configurations 1130 (e.g., which allowsconfiguration files and event journals to be backed-up and restored) andto check updates/download supplemental patches 1132 (e.g., which reviewscurrent version numbers of software, IR code databases, etc. and ensuresthat the latest versions are the ones available for download). It willbe appreciated by those skilled in the art that these functions may beperformed by software stored locally (e.g., on the PDA 1150 and/or hostcomputer 1152) or via the Web using, for example, Java applications.Optional desktop applications may also be used to edit 1134 and/oremulate 1136 the configuration and appearance of the PDA based remotecontrol application on the desktop system 1152.

Services and data that are located on a hosted site 1106 are representedby the remote control software services. The remote control softwareservices may include remote control application software updates 1140,supplemental IR database updates 1144 (including additional devicesand/or key functions for download), and/or a remote control usercommunity 1142 which, for example, might allow users to downloadadditional graphics for interface customization, upload theirconfiguration files to share with other users, etc. This user community1142 is also where third party developed software extensions may beplaced for download if desired. The hosted site 1106 may also be thesource of data that is used in conjunction with extended features 1146of the remote control application 1146, for example downloadableelectronic program guide (“EPG”) information, cable channel lineups,movie schedules and the like.

The remote control application software 1102 may also include thefollowing high level functions: setup wizards; consumer appliancecontrol through IR and/or RF signal transmissions; customizationfeatures including the ability to learn IR codes, assign and playbackmacros, reassign keys, program favorites including rapid assignment ofchannel numbers to network logos, configure multiple user profiles, setrestore points, configure a guest mode, customize the remote controlapplication from the host or remote computer including the downloadingof new IR codes and key functions, skins, additional network logos,software extensions; copying of configuration data from one PDA toanother; journaling activity; and an API for third party developers tobuild extensions. These functions will be described in more detail inthe following paragraphs.

Setup Wizards

The intent of a setup wizard is to get the PDA 1150 configured for useas quickly as possible. A setup wizard is normally a specialized part ofan application program that functions to guide a user through astep-by-step setup or configuration process. Information utilized inconnection with a setup wizard may be pre-populated so that there is anincreased likelihood of a user successfully getting the desired results.

By way of example, a user setup wizard serves to setup the remotecontrol application 1150 to identify one or more users and theirpreferences. For initial user setup, the user setup wizard may read datadirectly from the owner registration information file(s); no additionalinformation would be required to use the remote control application.Alternatively, for initial user setup, the user setup wizard need not beinvoked with the owner registration information file(s) being read andutilized by default.

By way of further example, a device setup wizard would be used toestablish the list of devices that the remote control application isconfigured to control as well as how the devices should be controlled.To this end, the graphical user interface of the remote controlapplication may start by displaying a small number of device buttons(e.g., buttons that would be activated to place the remote controlapplication into a mode to control a device) that would initially beun-setup. Selection of an un-setup device button can be used toautomatically invoke the device setup wizard. It is to be appreciatedthat the device setup wizard will be invoked each time the remotecontrol application is setup to control a new device or when it isdesired to add or change settings for an already setup device. Like thedevice setup wizard, the user setup wizard can be invoked multiple timesto add more users. When invoking a setup wizard, the remote controlapplication may be able to detect if a setup process has already beenexecuted and may display the current settings to the user forconfirmation or changing.

As will be described hereinafter, setup may prompt the user for the typeof install to perform (typical or custom). For custom setup, the usermay specify what level of device support is desired (standard, power,expanded). The setup wizard may chain together the functionality fromdifferent customization screens. Two major areas of customization areuser setup and device setup. The user may be required to run the setupwizards sequentially from the beginning or may select a specific setupwizard or portion of a setup wizard to run after initial configurationof the remote control application. The remote control application maystore user preferences in a configuration file and may recallconfiguration file preferences for inclusion in the setup wizard screensupon subsequent execution of those set wizards (e.g., in the performanceof an edit of a setup).

Consumer Appliance Control

Once the PDA 1150 has been setup, via the remote control application, tocontrol a device, the remote control application may be invoked to allowthe PDA 1150 to be used as the primary Infrared (IR) remote control forthe setup devices. In this regard, devices include conventional consumerappliances, such as televisions, VCRs, DVD players, PVRs, etc. Devicesmay also include home automation devices. While home automation devicesmay appear to be different from what might be viewed as conventionconsumer appliances (e.g., home theater equipment), from the standpointof the remote control application, they are quite similar. As long asthe home automation device can be controlled (e.g., via IR, RF,signals), the remote control application can cause the transmission ofsignals to the home automation device. Thus, the graphical userinterface of the remote control application may support controlsbuttons, sliders, etc., e.g., via the use of bitmap images, to supportoperations unique to home automation devices (e.g., lights dim/brighter,curtains open/close, fan rotation speed, etc.).

Remote Control Application Customization

Remote control application customization allows a user to modify thegraphical user interface, device and/or function control capabilities,etc, or the remote control application and/or PDA platform. Generally,any remote control application customization would be assisted by asetup wizard. Modifications achievable via customization include, by wayof example, learning IR codes, assigning and playing back macros,specifying favorites (assigning channel numbers and/or the IR sequencesnecessary to cause entertainment appliances to switch channels tonetwork logos), setting user security levels, etc. Before modificationsare made, an automatic backup of the settings of one or more componentparts of the remote control software application may be made so that theuser can undo changes if they did not achieve the desired results.

Remote Control Application Customization from PC/External Updates

Settings usable in connection with the remote control application (e.g.,skins, macros, etc.), remote control application updates, IR codeupdates, etc. may also be added, deleted, and/or modified based oninformation stored in external systems. The information may haveoriginated on a Website, the host computer 1152, etc.

Journal Activity

The system may be configured to journal (i.e., log) some or all of theactivities of the user(s) and/or the system into a file that may bestored on the PDA 1150. By way of example, some of the journal data maybe accessed to determine the most recently viewed channels, may be usedto generate and/or update favorites with an adaptive learning feature,etc. The journal data may also be made available for debugging purposes,for example by being synchronized with Web servers 1154 to acceleratethe speed with which a support team can debug issues.

Extensible through API for Third Party Developers

The system design may include an API that third party softwaredevelopers can use to add functionality to the remote controlapplication, communicate with remote control application, etc.

Electronic Program Guide (“EPG”)

With an EPG application on the PDA 1150 the user can view upcoming showsand choose which shows to watch without distracting anyone else watchingthe TV. In addition, the remote control application may work inconjunction with the EPG application to prompt a user to watch aprogram, allow them to command tuning to a program, record a program,etc.

Operating Environment

It is anticipated that the major operating environment for the PDA 1150would be the living room/home theater room in the home. However, theremote control application may be shipped with pre-configured lists ofdevices and device types associated with multiple rooms, including theoffice. The lists may be modified. Furthermore, a setup wizard can beused to configure the remote control application for one or moredistinct rooms.

Hardware Components

The basic hardware components and architecture of a PDA that serves as aplatform for the remote control application are well known in the art.Accordingly, for the sake of simplicity, only exemplary, select hardwarefeatures unique to the remote control functionality will be discussed indetail. Referring to FIG. 4, a standard PDA 200 has an IrDA transceiver202 used for short range communication with other PDAs and peripheraldevices. In addition, since the controllable devices generally uses anIR wavelength different than that used by IrDA, the exemplary platformis also equipped with an IR transmitter IC 204 and separate LED 206suitable for use with device IR control. In one embodiment, an IRsignaling module 1118 communicates with the IR transmitter IC 204through a serial connection 208 (for example COM1/COM4). Methods andcommand protocols for communicating with such an IR transmitter IC 204over a serial interface are well known and, therefore, need not bedescribed further herein (for example, see co-pending application Ser.No. 09/905,432). It will also be appreciated by those skilled in the artthat, while in the illustrated example the transmitter pulse timing iscontrolled by the specialized chip shown, in other embodiments it may bepossible to implement these timing functions via software running in themain CPU itself which would serve as a substitute for the IR transmitterIC 204. In addition, when adapted for use in connection with the remotecontrol application, the IrDA receiver may also be used for learning IRcodes from other devices such as originally supplied remote controls.The learning function would be made possible by paralleling the IrDAreceiver output signal 210 to both the normal PDA connection 212 and thelearning input 214 of the IR transmitter IC as shown.

The remote control application may also take advantage of any hard keysthat are provided on the PDA 1150. For example, the illustrated iPAQbrand PocketPC shown in FIG. 10 includes a navigation pad 104 and fouradditional keys 102 (“hotkeys”) at the bottom of the unit which arenormally used to facilitate rapid access to frequently used applicationssuch as a calendar application, address book application, etc. In oneexemplary embodiment, the remote control application may placeappropriate calls to the underlying operating system API to override thedefault hotkey functions and replace them with commonly-used remotecontrol functions (e.g., volume or channel adjust, etc.). Furthermore,key override may be configured to occur only when the remote controlapplication is the active or in-focus application. In this manner, ifthe user changes to another application, e.g., the Excel brandspreadsheet application, while the remote control application is stillrunning, the hardware hotkeys revert back to their previous settinguntil the remote control application is made active again. Additionally,the assignment of functionality to the hardware keys may be userconfigurable and/or automatically determined by the current mode ofoperation of the remote control application as described in more detailhereinafter.

The remote control application may also provide for the overriding ofany default shut-off timing. In this manner, the shut-off timing of thedisplay can be configured to be more appropriate for use in connectionwith a user's need to operate devices when the remote controlapplication is the program with “focus.” For example, by replacing thenormal shut-off timeout with a two-stage process such as: after 15seconds with no activity, switch to a splash screen then, after 5additional seconds, power down. Alternatively, these timeout values maybe made user-adjustable.

Software Components

Exemplary components of the remote control application 1102 are depictedin FIG. 5. The illustrated components comprise a presentation engine1108, a configuration engine 1110, a hardware communications manager1114 and a file I/O manager 1112. Each of these will be discussed inmore detail in the following sections.

Presentation Engine

The presentation engine 1108 manages the user interface that usersinteract with to command the operation of the remote controlapplication, for example, to cause the transmission of commands todevices. The presentation engine 1108 manages the mapping of the userconfiguration into the current hardware form factor—the iPAQ brand PDAin the exemplary embodiment, but in other embodiments this may be aWebPad or Tablet PC, a remote control software emulator, a remotecontrol itself, etc.

User Input

The user input module 510 handles all input events generated by theuser. This may include input from both the touch screen and hard keys.It may also handle input from a speech recognition software engine, forexample the IBM ViaVoice brand software installed by default on the iPAQ3950 brand PDA, or such other speech recognition applications as may beavailable. An action map 514 (see FIG. 18) controls all of the actionsthe user can perform for each device type. Thus, user input events wouldbe processed in terms of the action map 514.

Object Manager

The object manager module 511 manages resources used to create displayscreens. The object manager may also manage intrinsic controls as wellas controls developed by third parties.

Event Manager

The event manager module 512 may interface with an operating systemcalendaring function when events can be scheduled in the system.

Status Manager

The status manager module 513 may create a listening thread to captureany status broadcasts from devices. It may also track the current statusof the remote control interface application, e.g., to determine if it iscurrently on a control page, a favorites page, running a setup Wizard,etc.

Configuration Engine

The configuration engine module 1110 manages the configuration resourcesand creates display screens using the presentation engine module.Resources for the configuration engine 1110 are generally stored in aconfiguration file.

Resource Map

The resource map 521 may contain any bitmaps, string literals, pagelayouts, and other resources utilized in the remote control application.

IR Map

The IR map 523 contains any user-defined IR codes to be used by thesystem. These codes may have been learned or downloaded. The IR map mayalso contain links to an IR database associated with the IR transmitterIC and/or the main or supplementary memory system of the PDA 1150. IRmap links may be created at runtime to account for a possible change inmapping between IR database revisions.

Macro Map

The macro map 522 contains a list of the macros available within thesystem. Each control that executes a macro may refer to the macro map.

Device Map

The device map 520 manages the links between the different types ofinformation (resource map, IR map, and the macro map) for each appliancethe remote control application has been configured to interact with.FIG. 18 is an example of how the various maps relate to each other as auser performs an action, in this case a macro (turning the power on forthe home theater) as will be described in more detail hereinafter.

Hardware Communication Manager

The hardware communication manager module 1114 communicates with devicesexternal to the system utilizing the IR, Bluetooth, and 80211.b RFhardware, and possibly others. As new hardware is developed, access tosuch hardware may be through a specified interface within the hardwarecommunication manager. FIG. 6 depicts exemplary hardware communicationsinterfaces. Each of the interface objects is discussed briefly below.

Transmission

The transmission object 530 is utilized to control the output of a bytesequence via the required protocol stack (e.g., IR, RF or TCP).

Learning

The learning object 531 is utilized to control the learning of a bytesequence via the required protocol stack, for example data acquired viathe shared IR/IrDA input 214 to the IR transmitter IC 204.

Status

The status object 532 is utilized to listen for broadcast messages on aparticular protocol stack. This may be used for a two-way conversationbetween the PDA 1150 and other device hardware, e.g., hardware withinhome theatre and home automation systems.

Platform Hardware Integration

The platform hardware integration object 533 allows the remote controlapplication to utilize the particular hardware capabilities of the PDA1150 on which the remote control application is currently installed.

Protocol Stack

The protocol stack 537 is an abstraction layer between the differentprotocols used by the system to communicate with hardware within thehome. A basic version of the remote control application may only use theIR protocol stack 534, while more advanced (in capability) versions ofthe remote control application may use one or more additional protocolstacks (535, 536) either in conjunction with or instead of the IRprotocol stack.

IR

IR protocol 534 is used to communicate with any device using IR. The IRobject may interface with an IR transmitter IC 204 for transmission andlearning.

RF

RF protocol 535 is used to communicate with any devices using radiofrequencies. The RF component may be used to abstract the communicationbetween the remote control application and proprietary protocols, forexample, Bluetooth, Zigbee, 802.11, or other RF signals.

TCP

TCP 536 may be used more and more as the home becomes wired and devicesbecome connected via standard networking protocols such those used inconjunction with, for example, the IEEE 801.11b standard. The TCP stackmay allow the remote control application to work with any wired devicewithin the home, regardless of location. The TCP stack may also allowthe user to communicate with the home from a remote location via theInternet or other conduit.

The File I/O Manager

The file I/O manager module 1112 allows data access to be managed andcontrolled from a single place within the remote control application.FIG. 7 depicts the file I/O manager objects, each will be discussedbriefly below.

Parsers

The parsers 540 contain code to validate and parse the different fileformats used by the remote control application. The parsers may be usedmainly by the configuration engine 1110 to read and parse the differentresources required to display information.

Serialization/Persistence

The serialization/persistence object 541 is used by the remote controlapplication to read and write different file formats.

Device Configurations

The device configuration files 542 contain definitions of the devicesthe user has configured to be controlled via the remote controlapplication. Each file may describe the contents and layout of variousdisplayable pages for a device. These files may also contain anyresources used to command the operation of such devices. Withoutlimitation, the list of resources may include macros, IR codes, bitmaps,string literals and other data. A more detailed description of thesefile attributes are presented in later sections.

User Profiles

The user profiles 543 stores data about the different users of theremote control application which may include a user name, bitmap for auser avatar, user PIN, bit mask overlay and all preference settings. Theresults of adaptive learning (on a per-user basis) may be also stored ina user profile. Additionally, the user's favorite channel list (andassociated network logos) may be stored here as well.

User IR Code Database

The user IR code database 544 is used to store supplemental IR codesdownloaded from a Website or from some other source. The user IR codedatabase may be separate from the standard IR code database shipped withthe PDA 1150. The user IR code database may also contain learned IRcodes.

Event Journal

An event journal 545 may be provided to contain a log of user and systemactions that have occurred within the remote control application. Theevent journal may also contain broadcast messages received from deviceswithin the home. The event journal is a log of what the user has done aswell as what is happening within the home. It is contemplated that auser may be able to switch between a verbose mode and an alert mode. Theevent journal may also be used for debugging purposes.

System Features and Functions

The following paragraphs present detailed descriptions of individualfeatures of the remote control application. In some cases examples oftypical use are presented showing user input or other external action(s)or event(s) together with the associated response. It should beappreciated that such examples of typical use are intended to beillustrative only and, therefore, are not to be considered as limiting.

Setup Wizards

Setup wizards are parts of the application logic that guide the user ofthe remote control application through the steps to add devices to becontrolled, configure user preferences, etc. The results of using asetup wizard may be saved into configuration files 542, 543 to besubsequently referenced by the remote control application when needed.In the discussions that follow, setup wizards will be divided into twogeneral categories: user setup and device setup. A user setup wizard maybe invoked to identify user(s) of the remote control application andtheir preferences. A device setup wizard may be invoked to establish thelist of equipment that the remote control application may control and/orhow the equipment should be controlled. A device setup wizard may alsobe used to gather information about which devices the remote controlapplication may be operating. In this context, the overall initial setupprocess may be thought of as comprising four procedures:

(1) device identification;

(2) generation of macros (either via explicit entry or automatically);

(3) definition of favorites; and

(4) configuration of the home screen.

Setup wizards can be executed multiple times, for instance to add moreusers, add/change devices (to be controlled), add/change settings, etc.In an exemplary system, a setup wizard may detect if a particular setuphas already been executed and display the current settings to the userfor reference.

Platform Setup

Platform Setup—Device Identification.

To setup the remote control application platform, a device setup wizardmay be invoked to setup the type(s) of devices the PDA 1150 is intendedto operate (TV, DVD, DSS, AMP . . . ). This information may also includethe manufacturer of a target device. To this end the user may bepresented with one or more selections of IR codes and given theopportunity to test the different IR codes (i.e., cause transmission ofIR code signals) from a selected manufacturer in order to determinewhich IR codes cause a target device to respond. Once the user has foundand tested a successful combination, the device may be added to a listof devices available to be controlled.

Four different and well-known methods of device setup may be supported,namely, brand setup (described above); step and set (where the usertests the next IR code in a sequence of IR codes); direct entry (wherethe user enters a pre-determine code for the manufacturer/device typethat is the intended target) and downloaded Web codes (where the userdirects the remote control application to apply new IR codes gainedthrough internet access). In the event RF communications are utilizedbetween the PDA 1150 and devices, these same setup procedures may beutilized without limitation.

Once the correct command code set has been established by one of themethods described above, the remote control application may exposedifferent levels of functionality (i.e., keys to command devicefunctions) depending on the current user of the remote controlapplication. By way of example, the user interface may be implementedwith up to three levels of complexity. The default level is to exposestandard level functionality, (i.e., a representative set of the mostcommonly accessed command functions for the device type and modelconfigured). Alternative levels may include a power level, which exposesall command functions available within the remote control applicationlibrary for the device type and model configured, or a simplified level,in which only basic command functions are exposed to the user for thedevice type and model configured. These, or other levels, may be offeredto a user based on pre-set user parameters (e.g., as entered during usersetup described hereinafter.)

Platform Setup—Generation of Macros

The remote control application may analyze the remote controlapplication environment (i.e., the system comprising the setup devices)and, based on the knowledge of the user and the devices, the remotecontrol application may be used to create one or more macros to automaterepetitive device functions. By way of example, two common appliancefunctions, “Power Macro for Home Theater” and “Audio/Video Routing” aredescribed in more detail hereinafter, although it will be appreciatedthat many other such opportunities for automated or semi-automated tasksmay exist.

Power Macro for Home Theater

If all the appliances in a home theater system have discrete on/offcodes, the remote control application can offer a global (Home Theater)on and off function. To that end, the remote control application maydisplay a list of all available appliances (i.e., all appliances forwhich the remote control application has been setup to control) and theuser may select the one(s) to be involved in the Home Theater Powermacro. For example, the remote control application may present a listcomprising DSS, Cable Box, Amp, VCR, DVD and TV. The user may selectonly Amp, DVD and TV to be switched on and off in unison as part of aHome Theater Power macro since the VCR may be independently activated toreceive an input signal from the DSS and Cable box to record a program.

Audio/Video Routing

If the user routes all audio output via an amplifier, the remote controlapplication can assist the user by ensuring that the correct audio inputsignal is used when the user selects a device to be the device in focus.This form of macro setup may be accomplished by asking questions duringsetup that are utilized by the remote control application to associateinputs with devices. Alternatively, the macro setup may be accomplishedby utilizing information previously stored in Device Map 520 regardingdevice interconnections and input/output configurations. The macro(s)created as a result of the question/answer session may be customizableby the user to handle special situations.

Platform Setup—Favorites

The user may be able to specify a list of favorite channels for a numberof categories. The favorites lists may be synchronized with the channellineup offered by a cable or satellite service provider. During thesetup of personal favorites, a user may specify a group (of channels)that they always want to cycle through. The channels selected forpersonal favorites do not need to be in the same category (i.e., beavailable from the same device). For example, a news favorite mightinclude local broadcast channels while a movie favorite might include acable channel. A personal favorites group may include a unlimited numberof channels or may be limited in number to, for example, no more thanseven entries in order to balance the time consumed in stepping throughthe channel possibilities against the amount of program materialoffered.

Platform Setup—Home Screen

Another setup option may allow the user to customize the remote controlapplication home screen, an example of which is illustrated in FIG. 11.The remote control application home screen should not be confused withthe initial (splash) screen that is loaded when the user starts theremote control application. In this regard, the remote controlapplication home screen is the user's main portal to various devicecontrol screens or pages. During the setup of this home screen, the usermay select whether to display control through a device-centric view, anactivity-centric view or a mixture of the two, i.e., depending on theuser's preference the home screen may list the devices, or it may listthe activities that the user can perform, or it may list a combinationof both. It is to be appreciated that an activity represents a desiredconfiguration of one or more devices that is centered around a givenpastime. By way of example, a “watch movies” activity might be setup tocause the transmission of commands to select the DVD player as theaudio/video source, set the surround sound mode on the audio amplifier,switch the TV input to S-Video, etc.

Devices, activities, favorites, etc. that are not yet setup may bedisplayed on the remote control home page such that the iconicrepresentation of the device, activity, favorite, etc. has an overlay,for example a wrench as illustrated in FIG. 11. The overlay indicates tothe user that setup is required. In this regard, the icons function assoft keys that may be selected to cause the performance of a furtheraction, for example, to display a device control page, cause thetransmission of commands, etc. Additionally, if desired, the iconicrepresentation of an un-setup appliance, activity, favorite, etc. mayalso be displayed in a different color (e.g., “grayed” so as to bedisplayed fainter than normal as gray-scale image). The overlay and/orgraying may be removed once a setup procedure has been completed.

Setup may be automatically initiated in response to a user selecting adevice, activity, favorite, etc. that has not been previously setup,i.e. selecting an icon with an overlay as described above mayautomatically invoke a setup wizard corresponding to the type of theselected icon. Initial execution of the remote control application mayalso detect that setup has never been performed and automatically invokeone or more of setup wizards (which may be chained one to the next inthis instance) in order to establish a starting configuration for theremote control application. Still further, the remote controlapplication may be pre-configured with a set of typical appliances,activities, etc. This set may, for example, be the most popularbrand/model of each device type for the market or region in which thePDA 1150 was sold, may be a set of devices corresponding to a specificmanufacturer or brand associated with the PDA 1150, or some combinationof the above or any other criteria deemed appropriate.

Remote Control Application Start-Up and Home Page.

FIGS. 8 and 9 illustrate exemplary display screens with programsinstalled on an exemplary PDA 1150 including the remote controlapplication (labeled “Valhalla”). The user may initiate the remotecontrol application by, for example, tapping the Valhalla icon 82directly on the screen 80 or by selecting the Valhalla application 92from a drop down start menu 90. When the remote control application isstarted for the first time it may display a splash screen such asillustrated in FIG. 10. The splash screen may include information suchas product name and revision, company name, patent and copyrightnotices, etc. The splash screen may be OEM specific if desired.Alternatively, the remote control application, when staring from thisinitial state, may run a startup demo to show a quick overview offeatures of the remote control application. Once any splash screendisplay and/or demo are completed, the newly-started remote controlapplication may load a default home screen with a suggested set ofun-setup devices as illustrated in FIG. 11. Alternatively, an empty homescreen with only the set-up icon 112/113 shown in FIG. 12 may bedisplayed. Still further, the remote control application may bepre-configured to start with certain predetermined devices already setupto be controllable as noted previously, for example, those of aparticular manufacturer.

Icons representing devices and/or activities may be displayed in a wheel110 that rotates to bring the selected device or activity to the top ofthe wheel. The top (primary) icon 112 may be displayed larger (48×48pixels in an exemplary embodiment) than the other icons 113, 114, 115(24×24 pixels in said exemplary embodiment). Additionally, anintermediate size (36×36 pixels for example) may be provided for displayof secondary devices. The spacing of the icons may be dynamicallycalculated based on number of icons, to provide even spacing around thewheel 110.

In the example illustrated in FIG. 11, indication is made to a user thatdevices are in need of being setup, for example, by overlaying a wrenchsymbol over the standard device icon. Devices that are set up may float(i.e., be moved so as to be positioned) near to the top of the wheel110. By interacting with the wheel 110, the user may select a primarydevice 112 to initiate setup, or select another icon 114 to become theprimary device. When another icon 114 is selected, the wheel may performintelligent rotation turning in the direction that moves the desireddevice to the primary position—the wheel top—quickest.

The home page display shown by way of example in FIG. 11 or 12 may alsoinclude indication of the current user 116 and current room 118. As willbe described later, the icons representing the user and room may also beused to invoke configuration menus for these items. A default user namemay be taken from the PDA 1150 owner information. In the example shownin FIG. 11 (i.e., with suggested devices to setup already displayed onthe wheel, a default room is shown as the “Living Room” which includesthe devices: TV, DVD, Cable Box, VCR, Satellite, Audio Receiver).

The home page display may also include a KeyGuide icon 120 whichinitiates a display of the current assignments of the hard keys of thePDA 1150 and/or allows altering these assignments, an icon 122 toinitiate a customization menu, and icons 124 and 126 to switch to otherdisplay pages; all of which will be described in more detailhereinafter.

Device Setup

In order to control devices, the user may be required to first identifyto the remote control application the manufacturer and IR code used byeach device, as well as associating any desired special features(display name, macros, etc.) with the device. In general, device setupcomprises performance of one or more of the following steps:

1) The user selects a device icon to set up.

2) The user selects a device type. (In some cases, this may comprisesimply confirming that the device type desired matches the displayedicon that was selected.)

3) User specifies the device IR format and command set to be used tocontrol the selected device. This, in general, may occur in one ofseveral ways: the user may directly specify a desired setup code bynumber, the user may search through all codes available (local and/ordownloaded from Web Hosted Services), the user may select a brand namefrom a list presented by the remote control application (and may testvarious versions of that manufacturer's IR codes if multiplepossibilities exist), or may use such other technique as may beappropriate.

An exemplary setup process will be described in more detail in theparagraphs that follow.

Referring once again to FIG. 11, the device wheel 110 is shown with alldevices requiring setup. In addition, an add device icon 113 and addactivity icon 115 are presented. The device wheel 110 of FIG. 11represents an example of one starting view of the device wheel 110 whena user starts the remote control application for the first time. By wayof example, the steps involved in setting up a TV using a brand namewill be detailed. The user begins by touching the TV icon 112 (orpressing the power button 117 or selecting a hard key such as thenavigation pad 104, etc.). By way of explanation, during normaloperation (i.e., after the remote application has been setup for thedevice), touching the device icon would cause the remote controlapplication to display a page of control functions for that device (forexample, see FIG. 19 a). However, in this case, since the device has notyet been set up (as indicated by the small wrench icon overlaid on themain TV symbol), the setup wizard depicted in FIGS. 14 a through 14 i isinvoked. As an alternative (for example in the case where a TV devicehas already been set up and the user wishes to add a second TV, or wherethe user wishes to add a device not shown on the default wheel) the usermay select the add device icon 113.

The device setup wizard for TV setup may begin with an introductorymessage as shown in FIG. 14 a. Also, a keyboard 1401 and cancel 1412,back 1414 and next 1416 buttons, for use in connection with the setupwizard, may be displayed. In general, the keyboard and buttons can bedisplayed throughout each setup wizard page and occupy the same positionon all the screens for ease in use. Once the optional help message hasbeen read, the user would press the next button 1416 which steps to theexemplary screen shown in FIG. 14 b. The user may then select whichsetup method he wishes to use. In the example illustrated, the defaultsetup method—brand setup—is already selected and the user simply pressesthe next button 1416 to proceed to use this setup method. Other methodsof device setup, for example, code searching, direct entry, etc., arealso available and are well known in the art—see for example U.S. Pat.Nos. 4,959,810, 5,614,906 or 6,157,319 all of which are incorporatedherein by reference in their entirety.

To perform device setup in accordance with the step illustrated in FIG.14 c, the user selects the device type (TV in this example) from adrop-down menu 1418 of all device types supported by the remote controlapplication. Then, as illustrated in FIGS. 14 d and 14 e, the user maybe presented with a drop down list of brand names 1420 which correspondto the device type selected. The user may navigate this list using theup and down arrows 1422, slider 1424 displayed at side of the list, orany other conventional GUI device. Furthermore, for rapid positioning toa general area within the drop down list, the user may use the keyboard1410 to type in the starting letter(s) of a brand name. Once the brandname has been highlighted (e.g., “Sony” in FIG. 14 e), the user canindicate a desire to choose that brand by selecting (e.g., by tapping)the next button 1416. Additional GUI techniques for choosing a brand ofchoice are also contemplated.

If multiple IR code set possibilities exist for the brand selected, theuser may be presented with the opportunity to test each possible IR codeset using a drop down menu 1430 in connection with a displayed set oftest function keys 1432 as shown in FIG. 14 f. Test functions 1432 mayor may not be displayed with text labels indicating their designatedfunctions—the example 1432 shown in FIG. 14 f is without labels whilethe example shown in FIG. 20 b includes keys with text function labels.The user may activate one or more of the test function keys 1432 to seeif they function to control the device as desired (i.e., in response totheir activation an IR command code from the selected set is transmittedto test if the target device responds). If the target device fails torespond as anticipated, the user can simply select a different code setnumber from the drop down list 1430 and repeat the test procedure (e.g.,activate the test function keys again). When the correct setting isfound (i.e., the device responds as expected), the user can press thenext button 1416 to continue with the setup procedure.

Optional steps illustrated in FIGS. 14 g and 14 h allow the user toenter a tag name for the setup device. A default name, as shown in FIG.14 g, can be a simple concatenation of the brand name and the devicetype. The user may, however, elect to use a more descriptive name oftheir choosing, as shown in FIG. 14 h. Tapping next button 1416 may thenbe used to complete the device setup wizard process. The resultant TVdevice icon 142 would then be displayed on the wheel 110 (sans wrenchoverlay) together with the user-entered device name as illustrated inFIG. 14 i. Alternatively, an additional step may be included to allowthe user to select a customized icon to represent the device.

User Profiles

Exemplary steps for setting up a new user profile are illustrated inFIGS. 15 a through 15 f. With reference to FIG. 15 a, user customizationmay be initiated by tapping the current user icon 116 which results inthe display of a customize user menu 152. The customize user menu 152may be used to select a user—to place the remote control application ina mode to control devices, perform activities, etc. that have beenassociated with the selected user (in the example illustrated in FIG. 15only one user, “Pat,” is currently defined), to add/delete a user, oredit the profile of an existing user. By way of example, the stepsinvolved in adding a user will be illustrated. It will be appreciatedthat process of editing the profile of an existing user could be verysimilar.

A user setup wizard could be invoked by selecting “add user” from thecustomize user menu 152. The user setup wizard may commence bydisplaying an introductory message as illustrated in FIG. 15 b. The usersetup wizard may then prompt, as shown in FIG. 15 c, for entry of a newusername 1510 and, by way of example only, an optional password orpersonal identity number (PIN) 1520 which would be needed to authorizefuture changes to the profile being setup. In the example illustrated,this user setup wizard page also permits the new user to be defined as a“guest” via checkbox 1530. If set, guest status preferably disables allsetup and customization functions (device, user, room, favorites, etc.)within the remote control application while this user designated as aguest is active. Guest status may also be used to inhibit access toother applications of the PDA 1150 such as, by way of example, calendar,address book, spreadsheet, etc. The inhibit function may be implementedon a global basis—i.e., the guest user is prevented from exiting theremote control application—or may be on a per-application basis,depending on the level of support available within the PDA platform.

Once the name and any PIN or special status has been entered during usersetup, an icon can be selected to represent the new user (as illustratedin FIG. 15 d) and a sound may also be assigned to provide an audibleindication when switching the remote control application for use by thatuser (as illustrated in FIG. 15 e). To select an icon, the user mayscroll through a list (for example presented as a strip 1534) ofpredefined user images (which may alternatively be presented as adrop-down list, etc.), browse through bitmaps and select an image from afile folder within the PDA 1150, download an image, etc. Similarly, theuser may select a sound from a drop down list of predefined sounds,browse through all audio (e.g., .wav files) stored in the PDA 1150,download an audio file, etc.

To conclude the user setup process, a confirmation screen may bepresented, as illustrated in FIG. 15 f. Tapping the next button 1416 onthis confirmation screen may then return the remote control applicationto the home page display with the new user (e.g., the selected “Harley”icon shown, for example, FIG. 14 i or 16 a). The initialization of a newuser in the manner described above can also result in the creation of anew, blank favorites page that would be assigned to this user. Thepurpose and configuration of the favorites page will be discussed indetail in a later section.

Room Profiles

An exemplary process for configuring a room profile is illustrated inFIGS. 16 a through 16 f. Room profile configuration may be commenced,with reference to FIG. 16 a, by, for example, tapping the current roomicon 118 which results in the display of a customize room menu 162. Thecustomize room menu 162 allows the selection of a different room—toplace the remote control application in a mode to control devices,perform activities, etc. that have been associated with the selectedroom (in the example illustrated only one room, “Living Room,” iscurrently defined), the addition/deletion of a new room, or editing theprofile of an existing room. For configuration purposes, the customizeroom menu 162 also includes the option to display the device code setupnumbers for all devices currently configured in the selected room. Thedevice code setup information may be useful, for example, when accessingWeb-based services which provide extended function codes for devices,and are not supported by automated or semi-automated processes. (Foradditional background regarding semi-automated Web-based services, thereader is directed to co-pending application Ser. No. 10/151,501 “Systemand Method for Upgrading the Remote Control Functionality of a Device,”which is incorporated herein by reference in its entirety.) By way ofexample, the steps involved in adding a room will be illustrated. Itwill be appreciated, however, that editing an existing room might be asimilar process.

To invoke the room setup wizard, the “add room” item could be selectedfrom the room customization menu 162. Upon being invoked, the room setupwizard may begin by displaying an introductory message shown by way ofexample in FIG. 16 b. The room setup wizard may then prompt for theentry of a name 1620 for the room, as illustrated in FIG. 16 c, and animage to be associated with this room, as illustrated in FIG. 16 d. Toselect an image, the user may scroll through a list (e.g., a strip asshown, a drop-down list, or other-like GUI selection device) ofpredefined room images 1630, may browse through bitmap images in filefolders within the PDA 1150, download an image, etc. The user may alsooptionally associate a sound with switching to that room, as illustratedin FIG. 16 d. Again, the user may select a sound from a drop down listof predefined sounds, may browse through sound files (e.g. .wav files)stored in the file system of the PDA 1150, download an audio file, etc.

A confirmation screen may be presented to indicate that the room setupprocess is complete, as illustrated in FIG. 16 f. Tapping the nextbutton 1416 on this confirmation screen could then return the remotecontrol application to a home page display with the new room “Den”selected and an empty wheel 110 ready to be configured for the devicesand/or activities to be associated with the newly-defined room mode, asshown, for example, in FIG. 12. The initialization of a new room in themanner described above may also results in the creation of a new, blankfavorites page that would be assigned to that room (and user in thatroom if appropriate). The purpose and configuration of the favoritespage will be discussed in detail in a later section.

Room Setup—Phase 2—Generation of Macros

The remote control application may also provide for the automatic orsemi-automatic configuration of macros (pre-programmed sequences ofcommand actions which may be played back by pressing a single key) forcommon activities. By way of example, an often needed sequence ofcommands may be the powering on (and off) of several appliancestogether, such as may be case with a home theater system. To facilitatecreation of this type of exemplary macro, the remote control applicationmay, as part of a setup wizard, display to the user a list of allcurrently configured devices 1640 accompanied by “checkboxes” in whichthe user may indicate which of these are to participate in an “all on”macro. This may be included as an additional step in the room setupwizard, as illustrated in FIG. 16 g, since such macros will generally beassociated with the devices available in a particular room.

By way of further example, another often needed sequence of commands maybe those that perform input routing. To configure a macro to performinput routing, the user may answer a series of questions about themanner in which devices are hooked-up, may select a device hook-upconfiguration from a list (or from a list with visuals/diagrams), etc.Alternatively, where the underlying graphics capability of the device1150 supports it, a drag and drop interface may be provided to match theinputs and outputs of different devices that is representative of themanner in which the devices are hooked-up. Each of these input/outputpairs may then be used to establish a macro sequence to perform inputrouting so that, when a particular device is selected for use, all ofthe requisite audio/video controls are automatically selected or set toallow said use.

Customization of Home Screen

The system may allow the user to customize the appearance andfunctionality of the home page by, for example, adding a customizedbackground, displaying only devices or only activities on the wheel 110(the default may be to display both types, intermixed in the order inwhich they were set up), limiting access to setup functions, etc.

Initialization of the Remote Control Application and Interaction withOther Applications.

The remote control application may have basic behavior as it is turnedon and off. By way of example, an initialization sequence may beperformed when the remote control application is started. An overallview of a typical initialization sequence is shown in FIG. 24. In theillustrated example, advantage is taken of the hardware and softwareprovided by the iPAQ brand PDA to integrate seamlessly with the WindowsPocket PC 2002 brand software system. The basic initialization behaviorsof this exemplary platform are described in the following paragraphs,together with the interactions with other parts of the PocketPC brandsoftware system.

When a user selects the remote control application icon from the“programs” or “start” screen display (illustrated in FIGS. 8 and 9), aninternal consistency check is preferably performed to ensure that thefiles and resources needed to operate the remote control application areavailable. If a file is missing, for example, the user may be informedthat they need to re-install the file or an application. If problemspersist, a message may be displayed to inform a user that technicalsupport should be contacted. A dependency check may also validate thatthe required hardware (e.g., an IR signaling module 1118 or transmitterIC 204) is available and that it is the correct version for the remotecontrol application and other software currently installed. In the ClassInitialization, individual modules may be initialized to a default statebefore a configuration file is loaded. In the resource initialization,the base resources required to guide the user through the first timewizard may be loaded. The resources include the IR database, icons,bitmaps, dialogs, etc.

Thereafter, all required system resources are initialized, the remotecontrol application selects the appropriate start-up user and room (ifmultiples are configured). The system may check to see if the system ismulti-user or single user. If the system is single user, the multi userlogin may not be created. If the system is multi-user, the system mayload the user profiles (including user name, user icon, user PIN andwhich configuration file the may use). If the system is single user, thesystem may check to see if the user has one or more configuration files.If there is only one configuration file, then that configuration filemay be loaded. If the user has more than one configuration file on thesystem, the system may load the configuration file which is specified asthe default. The default may be the last configuration file used on thesystem. The system may set a bit so the system can provide a method toallow the user to (choose from and) load any other configuration files.If the first time user wizard flag is set, the system may gatherinformation about the user. If a configuration file is already on thePDA 1150, then the user may be prompted to use that one or create a newone.

Upon successful initialization, a splash screen may also be displayed,an example of which can be seen in FIG. 10. The splash screen mayinclude information such as the product name, company name, legalinformation, including patent and copyright notices, etc. This splashscreen may require user identification and/or log in or may simplyswitch to a home screen after a time out. In either case, on completionof the splash screen activity the remote control application would bethe application in focus. While active, the remote control applicationmay temporarily override the standard assignments of any hotkeys—asdescribed in more detail in other sections of this document.

When the remote control application is active, if the user performs noaction within a timeout period (e.g., 20 seconds which time may be usersettable) the system may turn off the display in order to conservebattery energy. However, even though the display is turned off, theapplication in focus may still be the remote control application. Thedisplay may be caused to briefly switch back from the home screen (orwhatever other screen is currently being displayed) to the splash screenas warning that a time out is about to occur.

If the user selects another application supported by the PDA 1150, forexample, a Pocket Excel brand spreadsheet application, the remotecontrol application may continue to execute in the background but wouldno longer be the application in focus. In this case, the remote controlapplication may be reinstated as the application in focus by the userselecting a hotkey (e.g., the iTask button), which hotkey would again beactive because the remote control application is not in focus, or byre-selecting the remote control application from the start menu orprograms page. In the case where the remote control application isreinstated, the remote control application may not perform the completeinitialization sequence described above. Rather, since the remotecontrol application is already loaded and active, it may simply resumeat the point at which it was suspended.

Favorites Page(s).

The remote control application may provide one or more favorites pagescontaining button icons which the user may configure for direct accessto his favorite programming (e.g., example, to cause the transmissionsof commands to cause a device to tune to a favorite channel), favoritedevice, etc. Access to such favorites pages may be, for example, via anicon 126 on the home screen page illustrated in FIG. 11. An exemplaryfavorites page is illustrated in FIG. 17 a. A favorites page may beoptionally pre-populated with a selection 1710 of commonly requestedcontent providers (e.g., using their logos) as shown in FIG. 17 a or mayinitially be blank. Initialization of the default favorites page ineither case preferably occurs on first start-up of the remote controlapplication. Separate, additional favorites pages may be created whenusers and/or rooms are added as described earlier. It will also beappreciated that favorites pages may be specific to particular devices(e.g. a satellite STB or a CD jukebox) while in other instancesfavorites pages may span multiple devices in order to allow access toall of a user's favorite media content from a single point of access.

In the case where the initial favorites page is pre-populated, iconswhich have not yet been configured by the user may be displayed with anindicia that setup is necessary (e.g., using a wrench overlay as seen inFIG. 17 a). In order to configure or add a favorites icon, the user mayeither invoke the customize favorites menu (instantiated, for example,by tapping the customize icon 122 while a favorites page is displayed)and selecting “Add Favorite” or “Edit Favorite,” as illustrated in FIG.17 b, or may simply tap one of the default icons that is displaying thewrench overlay to indicate that it is not yet configured. In eithercase, a favorites setup wizard may be invoked to configure a favoritesentry as described in the following paragraphs with reference to FIGS.17 c through 17 h.

The favorite setup wizard may begin by displaying the introductorymessage shown in FIG. 17 c. Next, the favorites setup wizard mayinstruct the user, as shown in FIG. 17 d, to record a sequence ofkeystrokes or other actions which are to be associated with thefavorites entry being configured. When the user taps the next button1416, the user may be presented with the normal set of device controlpages such as, for example, the satellite device control page shown inFIG. 17 e. The user may navigate amongst all the devices that have beenconfigured using the home page wheel 110 (the home page is accessed viaselecting icon 123) to switch to individual device control pages usingicon 124. By this means the user may enter a series of keystrokes, forexample, selecting the keys “0,” “0,” “4,” and “Enter” to assign to afavorites button commands for switching the satellite receiver deviceshown in FIGS. 17 e and 17 f to channel four. In an exemplary system, a“mini-wizard” (evidenced by the guidance messages 125) may continue toguide the user through the setup process while still providing fullaccess to the soft key pages within the application interface of eachdevice. When the user has completed entry of the desired sequence,tapping the customize icon 122 displays a menu which allows the user tocomplete the final steps of the favorites setup wizard. From thisexemplary menu, the user also has the option of temporarily suspendingand later resuming recording of keystrokes, inserting a delay into theplayback sequence, or testing the sequence entered so far by playing itback. The final steps of the favorites setup wizard, illustrated inFIGS. 17 g and 17 h, allow a name 1720 and image 1730 to be assigned tothe favorites entry. A sound may also be associated with selection ofthe favorite entry. Furthermore, where supported by the underlying PDAplatform, additional data such as, for example, a URL to be loadedautomatically into a concurrently running browser application, may beassociated with the favorite entry. (For additional background regardinguse of such data in conjunction with a user definable favorite key, thereader is directed to co-pending application Ser. No. 10/176,315entitled “System and Method for Retrieving Information While CommandingOperation of an Appliance,” which is incorporated herein by reference inits entirety.)

Navigation Model

The remote control application may take advantage of the hardware keysoffered on the PDA 1150 by redefining functions associated with thesekeys while the remote control application is active. By way of example,referring to FIG. 10, the exemplary iPAQ brand PocketPC devices offerfour hard keys 102 plus a navigation pad 104 (up/down/left/right andselect) arranged across the bottom of the unit. During normal operationthese key would typically be assigned by default to perform variousquick functions such as switching directly to a calendar or address bookapplications, displaying a list of currently active applications(sometimes referred to as using the “iTask button”), navigatingdirectories or menus displayed on the screen, etc. When the remotecontrol application is active, such hard keys may be defined to, amongother things, cause the remote control application to perform actionsthat are related to commonly used device control functions. Thisfunctionally may be further enhanced by allowing the user to selectbetween various groupings of hard key assignments based on their currentactivities. For example, optimum convenience during navigation of a menuon a PVR device may require that these keys be configured to cause thetransmission of commands for direction control, menu, exit, andselection functions; while optimum convenience while watching a DVD mayrequire that these keys be configured to cause the transmission ofcommands for transport control functions (play/pause/fast forward,etc.).

The various aspects of this hard key reassignment model and user controlover it will be discussed in the following paragraphs in conjunctionwith FIG. 13. These models include, but need not be limited to:

-   -   An exemplary application navigation model, which is driven by        the four hard keys 102;    -   Exemplary multiple navigation pad 104 key assignments, which can        be navigational, control, or transport;    -   An exemplary method for displaying a key guide to indicate        current key assignments; and    -   An exemplary method for selecting the active navigation pad key        assignments.

Application Navigation

Button 102 a can be used to cause the remote control application tocycle through each page type, e.g., home screen page, device controlpage (for a currently active device—the one at the top of the wheel), afavorites page, etc., as represented by the tab icons 123, 124, and 126,respectively, across the top of the screen illustrated in FIG. 11. Ifbutton 102 a is pressed when the application is active, but in low powerstate, the current page may be displayed. In this instance a secondpress may cycle to the next page type. Button 102 b can be used to causethe remote control application to display the next page for this pagetype (for example, control may be comprised of four pages, favoritescomprised of three pages, etc.). If button 102 b is pressed when theapplication is active, but in low power state, the current page may bedisplayed. In this instance a second press may cycle to the next pagewithin the family of pages for this page type.

Button 102 d can be used toggle mute (e.g., cause the transmission of amute toggle command to an appliance in focus). This button may be madealways available, regardless of the assignments selected for the otherkeys in the group.

Button 102 c can be used to display a key guide 130 and allow a user tochange key assignments for the navigation pad 104. When pressedinitially, the current key guide may be displayed, for example, overlaidover the bottom of whatever display page is currently active, asillustrated in FIG. 13. Display of the key guide 130 may be for a fixedor user-settable period of time, e.g. five seconds. If button 102 c ispressed again within this time window, i.e. while the key guide 130display is visible, navigation pad key assignment configurations can becycled through. By way of example, three possible selectableconfigurations 130 a, 130 b and 130 c are shown. The significance ofthese assignments is discussed in more detail in the followingparagraphs.

Navigation Pad Key Assignments

For user convenience, the five key functions present on the navigationpad 104 (up, down, left, right, and select) may be assigned to one ofseveral groups of function according to an activity currently beingperformed. This assignment may be changed dynamically and automaticallyby the remote control application according to the device type currentlyselected or activity currently being performed, or it may be manuallyset by the user. In the illustrated example, three sets of functions areavailable: navigation 130 a (for menu or guide operations on consumerappliances), control 130 b (volume and channel up/down), and transport130 c (play, rewind, fast forward, etc.). It will be understood thatactivation of the navigation pad 104 would then result in thetransmission of commands to control the appropriate operation of theintended target device.

An indication of the current setting of these keys is presented in thelower right corner of the screen by the key icon 120. This icon may takedifferent forms 120 a (indicating navigation), 120 b (indicatingcontrol) and 120 c (indicating transport) depending on the currentsetting of the navigation pad hard keys. Tapping this icon 120 mayinvoke a menu from which the desired key assignment can be selected bythe user. Alternatively, key assignments may be changed by use of thehard key 102 c as described earlier.

In the case where a user attempts to change the hard keys to a mode thatis not supported by the device selected for control, e.g., transportfunctions when the remote control application is in a mode to controlthe operation of a television, the remote control application canprohibit selection of that key assignment, can allow selection of thatassignment with the resulting activation of the hard keys resulting inthe no action being taken place by the system (or to cause transmissionof signals in a format appropriate for a default device or last devicecapable of performing the actions—punch through which may be manuallyset or automatically set), etc.

When the remote control application is currently in a mode where it isdisplaying a configuration menu or when it is executing a setup wizard,the navigation key pad 104 may automatically revert to standardapplication functionality, i.e., menu up/down/left/right and select, forthe duration of the setup wizard execution.

Control of Consumer Electronic Devices

The remote control application may be used to control the functionalityof a large number of consumer electronic devices. While a TV will beused as a template device for the illustrative description that follows,it will however be appreciated that many other device types exist.

Operation of a TV device is as follows. Assuming that the TV has beensetup as described earlier, the remote control application may be placedinto a mode for controlling the TV, for example, by rotating the homepage wheel 110 until the TV becomes the active (top) device icon—142 inFIG. 14 i—and then either tapping the TV icon 142 or the controls tabicon 124, resulting in the display of the first page of device controlpage as illustrated in FIG. 19 a. Depending on the device type,additional pages of control functions may be available as illustrated inFIGS. 19 b and 19 c. The user may navigate to these additional pages, ifany, by using either the page hard key 102 b or the control surfaceswitching icons 192. When the user selects a device as described above,the control page displayed may be the last one used, as opposed toalways beginning at page one. For example, if the last action a userpreviously performed before switching to another device type, such asthe VCR, was to command volume adjustment then, on re-selection of theTV, the third page (FIG. 19 c) might be displayed since this pagecontains the (most recently used) TV volume control buttons 194.

The soft keys displayed may be adjusted by the remote controlapplication according to the specific device that has been setup. By wayof example, FIGS. 21 a and 21 b show a control page as it may appear fortwo different television models. Of note, the Panasonic brand televisionmodel shown in FIG. 21 b supports a “-/--” key 1212, while the Sonybrand television model shown in FIG. 21 b does not. In this example, theremote control application displays only those buttons for functionsthat actually exist on the device currently selected.

While a control page is active (i.e., displayed), the customize icon 122may be used to invoke a menu 1202 which allows the user to make changesto key assignments and layout, learn new functions from another remotecontrol or device, add and delete keys and pages, all of which will bediscussed in further detail in sections to follow. Additionally, forconvenience, from this menu 1202 the user may optionally enable a textdisplay of key function labels as shown in FIG. 20 b. By way of furtherexplanation, the keys themselves are generally labeled using graphicalicons wherever possible in order to eliminate language dependencies (ascan be seen, for example, in FIGS. 20 a and 20 b), however, for userswho are not familiar with the icon system, optional text labels may beprovided.

Text labels may be generic functional names or may be the names assignedthat function by the manufacturer of the device which is currently beingcontrolled. By way of illustration, the remote control command functionwhich returns a TV set (or other tuning device) to the channel that waspreviously being viewed is usually referred to generically as “lastchannel” or “previous channel.” However, on Sony brand equipment thisfunction is often labeled “Jump,” on Panasonic brand equipment it isoften labeled “R-Tune,” etc. It will thus be appreciated that it may beadvantageous in certain applications to offer the user key labeling infamiliar terms.

As noted previously, the remote control application may displayfunctions that support a standard level of functionality (i.e., thosefunctions most commonly accessed), common power-related functions, orfunctions that are uncommon or extended functions that make devicesupport 100% complete.

Control of Home Automation Devices

The remote control application is in general capable of controlling anydevice that can be controlled using IR or RF communications whichincludes home automation devices. In the case of home automationdevices, the user may wish to have different button images to representthe functions and/or operation of these devices. Accordingly, additionalbutton images may be made available to the user, either stored locallyin the PDA 1150 or downloadable from, for example, a Web site or device,to help with the adoption and usage of the remote control applicationfor controlling these devices. Such a list of optional button icons mayinclude items such as on/off and open/closed icons for devices such aslights, curtains, blinds, air conditioning, heating, pool heater, spa,etc. The system may also have the ability to associate a function to animage, for example, to send a sequence of IR Codes to an X10 device.

Learning IR Codes

As new devices are introduced into the market, command sets to supportsuch new devices can be added to the remote control application. Thiscan be done one of three ways: learning, downloading, or occasionallyrefreshing the database that is burned onto the non-volatile memory ofthe PDA 1150. Detailed methods for learning IR codes are well known inthe art (for example, see U.S. Pat. Nos. 4,626,848 and 6,097,309 whichare incorporated herein by reference in their entirety) and the remotecontrol application may be provided with the ability to learn IR codesfrom a new device-specific remote control using one or more of these orlater-developed methods. To this end, the input signal to be learned maybe provided to an IR receiver specifically provided for this purpose ormay be derived from an output signal of an IR receiver 214 already builtinto the device 1150 as illustrated in FIG. 4.

Generally, IR codes are learned for a single function at a time. Theuser may choose to learn every available function or may elect to learnselect functions for that device type. The user can also choose toreplace the codes for existing functions with new IR code sequences.Additionally, the user can add a function that is not normally exposedfor this device type. When new functions are added, the user may add abutton that is to be assigned the learned command code(s) and providethe button with an appropriate label.

Generally, when the remote control application detects that learningmode has been entered, the software listens for new codes to learn. Thesoftware may respond audibly and/or visibly when codes are learned.Furthermore, the software may respond with a different audible toneand/or visible indication when codes are not able to be learned.

Assign and Playback Macros

Macros provide a way for the user to perform a combination of tasksquickly. There may be two types of macros supported by the remotecontrol application—system generated macros and user generated macros.System generated macros are generated automatically orsemi-automatically by the remote control application to facilitatecommon tasks like turning on and off the power to a home theatre system.System generated macros were described earlier.

User generated macros allow the user to manually program a sequence ofactions to be assigned to a single button such that that sequence can berepeated by a press of the single button. An example of a user generatedmacro might be a “Watch DVD Movie” macro, which: 1) turns on the DVDPlayer; 2) turns the AMP to the DVD input; 3) turns on the TV; 4) setsTV input to “Video 1;” and 5) plays the Movie. Such a user defined macromay be initiated in an exemplary embodiment via an activity icon on thehome screen device wheel 110.

With reference to FIGS. 22 a through 22 g, the user may initiateprogramming of a new macro sequence by rotating the add activity icon115 to the uppermost position of the wheel, for example by selecting it,as shown in FIG. 22 a. Selection of the add activity icon 115 may thencause an activity setup wizard to be invoked that guides the userthrough the programming process. The activity setup wizard may begin bydisplaying some informative messages as shown in FIGS. 22 b and 22 c.The activity setup wizard may then prompt the user to enter a sequenceof keystrokes to be stored as a macro. This entry process is similar tothat previously described in conjunction with FIGS. 17 e and 17 f(definition of a favorites key) and, therefore, need not be repeatedhere. Upon completion of the sequence entry, the user may select animage (FIG. 22 e) and name (FIG. 22 f) to associate with this programmedactivity. The user may also be provided with an option, for example viaa checkbox 224, to have the macro playback, on completion, either remainat the home page (box unchecked) or switched to the device control pageof the last device to be issued a command (box checked). This is usefulwhere, for example, a user programming a “Watch DVD” macro may wish thesequence to end with the remote control switched to the DVD devicecontrol page (such as shown FIG. 22 d) rather than the home screen page.When the macro definition is complete, the new activity (“Watch DVD” inthis example) appears on the device wheel 110 as shown in FIG. 22 g.Since similar images may be applicable to both devices and activities,activity icons may be displayed in a different color than device icons.In addition, the control button 117 in the center of the wheel may bechanged to a green “GO” button in place of a red “Power” button.

It is to be appreciated, that the software may allow a macro to beassigned to any screen that controls devices. Furthermore, the softwaremay allow macros to be added as punch-through commands for the hardwarehotkeys as well as common user interface elements. Still further, thesoftware may allow macros to be copied, linked, and/or edited. For usein editing, the software may allow a step-through capability to ensurethat the macro functions as desired and provide a macro editor so thatthe user can fine-tune the macro. Macros can also be recorded withnon-timed key strokes and/or with timed key strokes.

Customization and Reassignment of Keys

The remote control application may allow the user to exposefunctionality from one device on a control page that represents anotherdevice or simply rearrange the button layout of a given device controlpage. Customization and key reassignment may be accomplished by use ofthe copy key and move key functions accessible from the customizationmenu 1202 shown by way of example in FIG. 20 a. Using the customizationmenu 1202, the user may choose to override an existing button or add anew button to the screen. For example, copying a key function creates alink to the underlying functionality so, if the underlying behavior ofthe linked to key is modified, all linked keys would be updated.Alternatively, the user may have the option to change the default state(a link) to become a copy of the original functionality. In thisinstance, modifying the functionality of the “original” key would notaffect the other keys which were assigned the functionality of the“original” key.

In addition to redefining underlying key functionality as describedabove, the user may also be offered the option of adding new keys orediting existing keys. In this manner, key shapes and labels may bechanged as needed. The remote control application may offer a selectionof locally-stored key shapes and colors to choose from, others may bedownloaded from the desktop 1104 or Web hosted services 1106. By way ofexample, each button image may comprise two states, “Up” and “Down,” toenable a user to differentiate between an incative button and a pressedor activated button. The remote control application may also supportcustomization features such as are described in U.S. Pat. Nos.5,959,751, 5,515,052, and 6,014,092 which are incorporated herein byreference in their entirety.

Manage Favorites

Since each user may have different viewing habits and favorite channels,the remote control application may allow the favorites to be global (forall users) or specific to a user. For example, in a household withseveral people the channel lineup will remain the same, however thefavorites for an individual may vary. Furthermore, since favorites mayactually perform several functions (associating a readily identifiablelogo with a network channel and/or sending IR or other commands todirect a device to the proper channel), it may be of use for oneindividual to set up the entire network lineup as favorites, and allowother users to copy the entire set of favorites pages. Each user maythen further customize their set of favorites pages by eliminating thosethat are not normally viewed, or re-arranging the favorites onto pagesto their liking. This technique of eliminating favorites may also beused, for example, by a parent when configuring an acceptable channellineup for a child. The child's user profile could be set up with a PINthat the child doesn't know (and thus has no ability to configure)and/or set up as a Guest so that other configuration changes may not bemade. There may also be a setting in the interface to only allow (on aper-user basis) channel access through favorites (and not by channelup/down or numeric keypad entry) to further limit the channel lineupassociated to particular individuals.

Assistance may be offered during the setup of favorites describedearlier in conjunction with FIG. 17. For example, a short list (sevenentries, for example) may be maintained by the remote controlapplication which represents a user's most frequently accessed channels.During execution of the favorites setup wizard, the current list offrequently accessed channels may appear thereby allowing rapid,semi-automatic programming of favorites buttons effected by pickingchannels from the displayed list. Optionally, the user may be able tocreate multiple such lists to cover a variety of genres or categoriesbuilt into the favorites setup wizard.

When the remote control application is adapted to be used in connectionwith EPG functionality, the system may read extended channel informationprovided in the EPG to automatically group the channel into categories.Genre categories may also be supported, for example Personal, News,Sports, Movies, Music, etc. The user may be able to create additionalcategories with custom names. The user may be able to add a channelnumber (with device) to a category. The user may be to add a label tothe channel. The user may be able to associate a logo with the channel.The system may handle favorite channels implemented across multipleinput sources. For example, the movies list may contain one local cablechannel and four channels supported by satellite. The remote controlapplication may also monitor the EPG data periodically downloaded fromthe hosted services site 1106 for changes in a channel line-up tothereby adjust favorites channel programming sequences eitherautomatically or by user option.

The favorites page display may also be adaptive. In this regard, theremote control application, on an user by user basis, may organize thedisplay of the favorites page as a function of frequency of channelaccess. The most oft selected channels may be displayed on the top ofthe list. This display order may also change based on day of week andtime of day, either automatically or by user request.

Extensible File Format

User IR Code DB

The user IR code database may be used to store supplemental IR Codes asthey are downloaded from the website or from some other sources. This isseparate from the standard IR code database shipped with the PDA 1150.The user IR database may or may not contain learned codes. It is desiredthat all learned codes are accessible to all users of the remote controlapplication.

Device Configuration File

The device configuration file contains the devices the user hasconfigured to control with the remote control application. The deviceconfiguration file may also describe the contents and layout of thedevice control pages of the remote control application for each device.The device configuration file may additionally contain any resourcesrequired by the individual device. The list of resources include, butare not limited to, macros, IR codes, bitmaps, string literals and otherrequired data.

Extensibility of Configuration Files

In order to manage the configuration and user set-up data associatedwith the remote control application as described in the precedingsections, it is advantageous to store this information in a file systemwhich is designed to facilitate easy future extensions to theinformation stored as well as transfer of this information between likedevices. An exemplary file structure which may be used to achieve thesegoals is presented in the following paragraphs and Tables 1 through 12below.

Configuration Header

As illustrated in exemplary Table 1, the configuration header containsdata fields for use in describing attributes of the file.

TABLE 1 Attribute Description Type Size RecordSize Size of this recordUSHORT PageWidth Width of a page Short PageHeight Height of a page ShortFileVersion Version of this file format BYTE 8 PlatformID_Key ID to arecord in the Long Platform Table ArchiveFlag Specifies when thedocument Long is to be archived.

Platform & Hardware Table

As illustrated in exemplary Table 2, the platform & hardware tablecontains data fields for use in describing attributes of the hardwareplatform on which the file was generated.

TABLE 2 Attribute Description Type Size RecordSize Size of this recordUSHORT PlatformID_Key Unique ID for this device Long ScreenWidth ScreenWidth Short ScreenHeight Screen Height Short HardwareKeyMap_ID HardwareKey Map UINT PADSize Size of Paddings Short PAD Padding . . . BYTEvariable

Device Table

As illustrated in exemplary Table 3, the device table contains datafields for use in describing the attributes of each of the devices theuser has setup the remote control application to control.

TABLE 3 Attribute Description Type Size RecordSize Size of this recordUSHORT DeviceID_Key Unique ID for this device Long LabelLength Length ofthe string label Short Label String literal for this device Charvariable DeviceType Class of device this device UINT header refers toSetupCode Code for this particular Device UINT BrandNameLength Length ofthe BrandName Short string BrandName String literal of this brand Charvariable ModelLength Length of Model string Short Model String literalof this Model Char variable DeviceSpecificDataSize Size of theDeviceSpecificData Short DeviceSpecificDataConfig DeviceSpecificDataBYTE variable DownloadDeviceDataSize Size of the downloaded data ShortDownloadDeviceData Downloaded data BYTE variable OriginFlag Who createdthis record UINT PADSize Size of Paddings Short PAD Padding . . . BYTEvariableExternal downloaded data will be stored in DownloadDeviceData.

Page Table

As illustrated in exemplary Table 4, the page table contains data fieldsfor use in describing the attributes of each display page of the remotecontrol application.

TABLE 4 Attribute Description Type Size RecordSize Size of this recordUSHORT PageID_Key Unique ID for this page Long DeviceID_Key ParentDevice ID Long BackgroundID_Key ID to a resource record Long PageTypeDescribes what this page UINT will contain - Home Page, Power, HomeTheatre, Device Page, Favorite Page BackgroundFillRGB RGB Fill ColorCOLORREF OriginFlag Who created this record UINT PADSize Size ofPaddings Short PAD Padding . . . BYTE variablePage records are preferably grouped by device ID.COLORREF is preferably a 32-bit value used to specify an RGB color.

Label Page Table

As illustrated in exemplary Table 5, the label page table contains datafields for use in describing the labels within a page.

TABLE 5 Attribute Description Type Size RecordSize Size of this recordUSHORT LabelID_Key Unique ID for this Label Long PageID_Key Parent PageID Long LabelLength Length of label string Short Label String LiteralChar variable LocationX X Location UINT LocationY Y Location UINTLocationZ Z Location UINT Width Width of Label UINT Height Height ofLabel UINT Font Font Type UINT FontSize Font Size UCHAR FontAlignmentFont Alignment UCHAR (Center/Left/Right) TextColor RGB Color of the TextCOLORREF

Border Table

As illustrated in exemplary Table 6, the border table contains datafields for use in describing page borders.

TABLE 6 Attribute Description Type Size RecordSize Size of this recordUSHORT BorderID_Key ID to a resource Long record PageID_Key Parent PageID Long LocationX X Location UINT LocationY Y Location UINT LocationZ ZLocation UINT BorderFillRGB RGB Color of COLORREF the Fill

Key Table

As illustrated in exemplary Table 7, the key table contains data fieldsfor use in describing attributes of keys supported by the remote controlapplication.

TABLE 7 Attribute Description Type Size RecordSize Size of this recordUSHORT KeyID_Key Unique ID for this Key Long PageID_Key Parent Page IDLong LabelLength Length of key label Short string Label String LiteralChar variable LocationX X Location UINT LocationY Y Location UINTLocationZ Z Location UINT ImageUpID_Key ID to a resource record LongImageDownID_Key ID to a resource record Long ImageDisabledID_Key ID to aresource record Long SoundDownID_Key ID to a resource record LongSoundUPID_Key ID to a resource record Long Status Status of key UINTFunctionID_Key ID to a Function Long record MacroID_Key ID to a set ofLong macro records FontType Font Type UINT FontSize Font Size UCHARFontColor Font Color COLORREF FontAlignment Font Alignment UCHAR KeyTypeMoveable Y/N BOOL KeyPlacement Hardkey/Softkey Y/N BOOL ValidityDay Daysthe key is valid Long ValidityTime Time the key is valid Long OriginFlagWho created this UINT record (User, Dynamic, . . . ) PADSize Size ofPaddings Short PAD Padding . . . BYTE variableKey records are grouped and sorted by PageID_Key.KeyType is a boolean to specify the key is movable or not.

Function Table

As illustrated in exemplary Table 8, the function table contains datafields for describing functions performed in response to activation of akey.

TABLE 8 Attribute Description Type Size RecordSize Size of this recordUSHORT FunctionID_Key Unique ID for this page Long FunctionNameLengthLength of FunctionName string Short FunctionName String Literal of thefunction Name Char variable HardwareType Hardware Type that implementsthis UINT function DeviceType Class of device this device header UINTrefers to. DeviceID UEI ID for this Device UINT DeviceID_Key ID toDevice Long LearnedKeySize Size of learned data Short LearnedKey Blockof learned Key Data BYTE variable MagicCode UEI Magic Code UINT KeyCodeID to a resource record Long HardwarePayloadSize Size of HardwarePayloaddata Short HardwarePayload Hardware specific data stream BYTE variableMacroID_Key ID to a set of macro records Long PageID_Key Page to bejumped to Long OriginFlag Who created this record UINT PADSize Size ofPaddings Short PAD Padding . . . BYTE variableHardwareType may be, for example, RF or IR transmitter IC.If the ID function is in the downloaded data, then DeviceID_Key ispointing to the device; otherwise, it's NULL.

Macro Table

As illustrated in exemplary Table 9, the macro table contains datafields for describing attributes of a macro function.

TABLE 9 Attribute Description Type Size RecordSize Size of this recordUSHORT MacroID_Key ID for this macro set Long FunctionID_Key ID to afunction record to Long execute NextMacroID_Key ID to a Next MacroRecord to Long execute KeyDownTime The amount of time is UINTmilliseconds the key stays down InterKeyDelay Delay amount after UINTexecution in milliseconds OriginFlag Who created this record UINTPADSize Size of Paddings Short PAD Padding . . . BYTE variableAny nested macros will be copy entire functions from the macro list tothe new macro list.

Resource Table

As illustrated in exemplary table 10, the resource table contains datafields for defining resources (e.g., image file, sound file, etc.)

TABLE 10 Attribute Description Type Size RecordSize Size of this recordUSHORT ResourceID_Key Unique ID of a resource Long ResourceType Type ofresource stored (Bitmap, UINT Wave, . . . etc) ResourceDataSize Size ofthe resource data Long ResourceData Byte stream of the resource BYTEvariable ResourceWidth Width UINT ResourceHeight Height UINT PADSizeSize of Paddings Short PAD Padding . . . BYTE variable

Event Journal File

As illustrated in exemplary Table 11, the event journal table containsdata fields for describing events captured during the course ofoperation of the remote control application.

TABLE 11 Attribute Description Type Size RecordSize Size of this recordUSHORT EventID_Key Unique ID of this Event Long EventType Type of Event,System or User Long EventTime Time of the event Date/time EventDataSizeSize of EventData Short EventData Data of the event BYTE variablePADSize Size of Paddings Short PAD Padding . . . BYTE variable

User Profile File

As illustrated in exemplary Table 12, the user profile file containsdata fields for use in describing user attributes.

TABLE 12 Attribute Description Type Size RecordSize Size of this recordUSHORT UserProfileID_Key Unique ID of this User Profile LongUserNameLength Length of UserName Short UserName Name of the User Charvariable UserAvatarID_Key ID to a resource ID Long UserPin 4 digit PIN.(Or minimum UINT 4 password length as recommended/required by theoperating system software.) UserTimeOut Time before the user is loggedout UserRole Role of the User UserRating V-Chip Rating for the UserPADSize Size of Paddings Short PAD Padding . . . BYTE variable

Room Profile File

As illustrated in exemplary Table 13, the room profile file containsdata fields for use in describing room attributes.

TABLE 13 Attribute Description Type Size RecordSize Size of this recordUSHORT RoomID_Key Unique ID of this Room Long Profile RoomNameLengthLength of RoomName Short RoomName Name of the Room Char variable LocaleLocation of Room UINT PADSize Size of Paddings Short PAD Padding . . .BYTE variable

Functionally, it is to be appreciated that file formats may beversionable, file formats may be backwards compatible, file formats maybe extensible to include new features, file formats may allow thirdparty developers to use the storage mechanism, file formats may providean efficient storage model, and files may reduce duplication ofexpensive resources like bitmaps.

Configuration Files to Support Multiple User Profiles

The remote control application is preferably designed to be used bymultiple users. Accordingly, a user profile file may store data about auser of the system including the user name, bitmap for the user avatar,user PIN or password and all preference settings. In addition, the shortlist of most frequently accessed channels on a per-user basis may bestored in the user profile. The user's favorite channel list (andassociated network logos) may be stored in the user profile as well.

In certain instances, it may be preferred to designate a primary userthat controls the look and functionality of the remote controlapplication. All the other user profiles may be derived from the primaryuser's user profile (the “master profile”). Any changes in the masterprofile may be updated in any of the profiles that are based from themaster profile. Thus, the master profile may be fashioned after atemplate where any documents using the template may be updated if thetemplate is modified.

Once the primary user has setup the main configuration, newconfigurations can be created based on the master profile. The primaryuser may choose which devices and which levels of functionality toexpose. This can include backgrounds, specific favorites and otherfeatures. The primary user may also choose whether the profile is to be“locked down,” which may be used to prevent any additional customizationby other users of the profile. In this manner, the primary user canmaintain complete control over the way other users (e.g., familymembers) interact with the remote control application.

All adaptive learning functions may be stored within each profile sothat the individual users can access their favorite channels rather thananyone else's. Even if a profile is “locked down,” favorites may bestored on a per-user basis. Additional facilities may be needed to givethe primary user the ability to review the viewing habits of otherfamily members.

For personalization, icons (or avatars) may represent each user. Theavatar may be a 32×32 bitmap stored in the user profile. When the remotecontrol applications starts, a splash screen may be generated with thebitmaps to allow the current user to select an appropriate user avatarand, thereby, gain access the remote control application configurationassociated with the selected user.

User Profile

The following Table 14 contains exemplary attributes of the user profileobject type.

TABLE 14 Attribute Description User Profile ID Primary Key - Unique IDof this User Profile User Name Name of the User User Avatar File PathFile location to the Avatar User Avatar Bitmap Data Avatar bitmap dataUser Avatar Bitmap Data Size Size of the Avatar bitmap data User AvatarWidth Width of the Avatar Bitmap User Avatar Height Height of the AvatarBitmap User Nickname Nickname for the user User Pin 4 digit PIN

Favorite Map

The following tables contain exemplary attributes of the favorite mapobject type.

TABLE 15 Attribute Description Favorite ID Primary Key - ID to thisFavorite Device ID Foreign Key - ID to the Device which supports thischannel favorite - Satellite, Cable Box, Tivo or VCR. Channel # Thechannel # for the favorite Channel Name The channel name of the favoriteNetwork 4 Letter identifier The channel 4 letter identifier. NetworkLogo Bitmap ID Foreign Key - ID to the Bitmap table stored in theconfiguration file User Network Logo File Path File location to theNetwork Logo User Network Logo Network Logo bitmap data Bitmap Data UserNetwork Logo Size of the Network Logo bitmap data Bitmap Data Size UserNetwork Logo Width Width of the Network Logo Bitmap User Network LogoHeight Height of the Network Logo Bitmap

Thus, the remote control application may store information regarding auser's preferences in a configuration file. The user profile may storethe user name, bitmap for the user avatar, user PIN and all preferencesettings, including favorites, network logos and the results of adaptivelearning. Since the system may support multiple configuration files peruser, the system may display content using (by default) the user's lastused configuration file. The system may also allow the user to select adifferent configuration file to use. The system may allow multiple usersto store their configuration files on the same PDA 1150. The system maystore the identity of last user that used remote control application.The system may require that one of the users be named as a primary user.The configurations set in the master profile for the primary user may beavailable for use in establishing configurations used by other users.Any changes made to the master profile may be available to theconfigurations used by other users. The system may allow the primaryuser to determine the level of “lock down” associated with one or moreuser configurations which may, in turn, determine how secondary userscan customize their settings.

Restore Point

Whenever the user performs an operation that may change theconfiguration of the remote control application, the remote controlapplication may create a restore point just before making the change tothe configuration. For example, the system may create a restore pointwhen the user changes the configuration file, changes user settings, orchanges the IR code database. Thus, the restore point maintains settingsthe remote control application may revert to if the user decides toreturn to a state prior to a change.

The restore point may be reset if the user chooses to save the newconfiguration and update the restore point, for example, by interactingwith a check box in a save dialog. Preferably, the supplemental IRdatabase is backed up using a restore point when the remote controlapplication learns new IR codes. The supplemental IR database is alsopreferably backed up when the user applies a new IR database patch.Generally, the configuration file may be backed up on local storage.However, if no local storage is available, the configuration file can beuploaded to a PC the next time the device 1150 is docked. To this end,the user may be prompted to dock the device after making changes and theuser may be continually prompted to dock upon startup of the remotecontrol application until it is detected that the configuration file hasbeen successfully backed up. When docked, the configuration file may beautomatically backed up if anything has changed. A CRC may be stored inthe file after changes. If the CRC changes between synchs, the file maybe backed up (given that space is available).

Guest Mode

The primary user may elect to allow guest access (created as anotheruser profile) which allows the guest user to operate the home system ina simple way, e.g., no learning functions are available, there is noaccess to other programs on the device 1150 without PIN confirmation,etc. Furthermore, guest access may override the ability to power downand reboot the system as a means to circumvent the guest mode. The guestprofile may be based on the master profile as a base configuration andthe guest may have no ability to change the configuration.

Security

For a user, the most sensitive information in remote control application(without support for E-Commerce transactions) is generally considered tobe the data stored in the event journal. For the manufacturer, the mostsensitive information is generally considered to be the IR codedatabase. Therefore file encryption may be applied to the event journaland the IR code database as a measure of security.

The user may have other sensitive data stored on the PDA 1150,including, but not limited to: schedules, contacts, financialinformation, notes, and email. The user needs to feel confident that thedevice 1150 used to hold this confidential information can sit on ornear the coffee table for hours in the evening, with others around. Thislevel of confidence may be provided by limiting access to only theremote control application functions when the remote control applicationis executing unless a PIN code is entered. Furthermore, the primary usermay elect that no additional named users are allowed and that guestaccess is denied. The primary user may enter a PIN for securetransactions, including starting the remote control application. Thisputs the remote control application in a mode that no other programs canbe accessed without a PIN code to thereby render all external data safe.

If the primary user elects to add secondary named users, they may alsohave PIN codes assigned. Secondary named users may be able to learnfunctions (if the primary user has not “locked down” the configuration)and store favorites. However, there may be no access by secondary usersto other applications or files on the PDA 1150 without PIN confirmation.

Quick Assignment of Channel Number to Network Logo

Since the cable and satellite companies are constantly refining theirchannel line ups, the remote control application should provide a simpleway to modify the channel numbers that the user assigned to favorites.Such channel number reassignment may be accomplished by, for example,the user holding down a channel logo (i.e., a button used to tune to achannel) for a given time (e.g., 3 seconds—as opposed to tapping it,which would normally execute the channel selection or other sequenceassigned to the key) to invoke a setup wizard similar to that previouslydescribed in connection with FIGS. 17 a through 17 h.

Alternatively, the user may specify a channel number as a numericvariable instead of a series of keystrokes. This number is thenconverted by the remote control application into an appropriate sequenceof IR command signals at time of execution. For additional backgroundregarding methods for specifying and converting numeric variables intoremote control IR signals, the reader is directed to co-pendingapplication Ser. No. ______ entitled “System and Method for Interactingwith a Program Guide Displayed on a Portable Electronic Device” of likeassignee, which is incorporated herein by reference in its entirety.

Besides simplifying user definition of favorites, in the case of aremote control application that supports EPG functionality this approachcan also keep the favorite channels (and perhaps others) in synch bymonitoring the channel listings from the EPG. In this case, when an EPGis downloaded, typically on a periodic basis, the remote controlapplication may compare the most recently downloaded EPG against apreviously downloaded EPG to determine if any channel assignments havechanged. If any channel assignments have changed, the remote controlapplication may then check the favorites list to determine if any of thechanged channels are listed therein. If so, the remote controlapplication may update the favorites channel list to reflect the newchannel number assigned to a favorite. By way of example, assume that anEPG comparison indicates that “ESPN” has changed from channel 29 tochannel 45. If “ESPN” is in the favorites list, the favorites list willbe updated to change the mapping of “ESPN” from channel 29 to channel 45whereby subsequent activation by the user of the “ESPN” logo icon willcause the transmission of command(s) to tune the target device to thecorrect channel assignment.

It is also contemplated that the remote control application may storefavorite channel assignments as broadcast channel identifiers and usethe data in the EPG to dynamically convert these channel identifiers toabsolute channel numbers each time the favorite is selected by the user.In this case, in keeping with the previous example, a favorite would bestored simply as “ESPN” which would be translated to channel 29 orchannel 45, as appropriate, based on the EPG data currently loaded atthe time the “ESPN” logo icon is activated (i.e., the channel thecurrent EPG shows “ESPN” is being broadcast on). In this manner, therewould be a late binding between the “ESPN” favorite and the channel onwhich “ESPN” is being broadcast.

When logo icons or other icons are activated, different states may bedisplayed to the user. For example, the system may support a networklogo in the form of two bitmaps corresponding to a button up state and abutton down state. Logo dimensions and color depth to be used may besupplier specified and may vary for the different platforms. The systemmay also store network logos in the user profile, since the favoritechannel information may be there.

Download Skins

Skins are manufacturer or user developed bitmaps to represent buttons,backgrounds, etc. A button can have multiple states, up, down anddisabled. The disabled state may be used to support functionality wheretwo-way communication between the target device and the remote controlis possible, where a particular user profile has no access to a function(for example, because it has been disabled by the primary user), etc.

Configuration files containing bitmap resources may be made available aspart of vendor-supplied extended features offered on Web-hosted extendedfeatures service 1146 or may be uploaded by users themselves to a Webhosted user community 1142 for downloaded by other users. Suchconfiguration files may be made available for specific devices or mayrepresent generic keyshapes, etc., suitable for use with any device. Byway of example, the system may look in a default named local directoryfor configuration files and in a default named directory on the host PCfor configuration files to be updated during ActiveSync. When updated,the system may override bitmaps from previous configuration file.

Copy Configuration from One PDA to Another

The ability to share configuration files allows for growth in usage ofthe remote control application. A configuration file can be shared in anumber of ways. It could be uploaded to an Internet site, sent via mail,or copied on to a floppy disk. The user who receives a configurationfile may use the ActiveSync application to load it onto a PDA 1150.Another method is that a user could transmit (referred to as “beaming”)the configuration via the IRDA port on one PDA to another PDA. Thismethod takes advantage of facilities available on the typical PDAhardware platforms.

The configuration file can be broken down into elements, for exampledevices, pages, themes, learned IR codes, network logo icons andfavorites. The user may wish to share only a few elements or the wholeconfiguration file.

The configuration files are preferred to have versioning, so that if auser shares a configuration file with a user who has an older version ofthe remote control application, the transferred configuration file ishandled gracefully. If the configuration file has a dependency on a newversion of an IR database module, then the user may be notified that thedevice may not function properly until the user updates their IRdatabase. The system may be able to check version number of thesoftware, version number of the IR Transmitter IC installed, and/ordetermine the form factor (PDA or WebPad) and the OEM of the platform.

The sending user may select from a list of options what they want tobeam. The receiver may choose whether the received configuration file ismerged with an existing configuration file or stored as a newconfiguration file. If the configuration file is merged, the remotecontrol application may ask the user if the would like to update theirhome theatre mode settings. The receiving system may determine thedestination location for the beamed or downloaded resources.Furthermore, users may be able to copy resources related to aconfiguration file to a hosted Website to enable sharing on a globalbasis. Thus, unlike most current home universal remote controls thatonly support a single configuration, the system may provide a mechanismto allow users to have multiple configurations to choose from.

Download Network Logos

A network logo is a graphical representation of a network station orchannel. In its base form, it is a button with an up bitmap and a downbitmap. A disabled bitmap is not needed since the button does not haveany state. The download network logos feature represents a set ofresources that can be accessed through the Internet. To implement thisfeature, almost all of the architecture modules come into play: hostedremote control software services, desktop remote control softwareapplications, and remote control software. Network logo sharing mayallow users to transmit the network logo information button to anotheruser, where the receiving user could chose to add it to the favoritelist, channel lineup page, or the like. The system may operate inconnection with a mechanism to establish access to network logo bitmapsfrom technology partners such as cable operators or EPG providers.Network logos may also be made available from a hosted Web site. Thesystem may allow users to share network logos through IR beaming.

Journal Activity

The event journal may contain a list of user and system actions thathave occurred within the remote control application. The event journalmay also contain broadcast messages from devices within the home. Theevent journal is a log of what the user has done as well as what ishappening within the home. The user may be able to switch between averbose mode and an alert mode for the broadcast messages. The eventjournal may also be used for debugging purposes.

An exemplary event journal entry format is shown in Table 16.

TABLE 16 Attribute Description Event ID Primary Key - Unique ID of thisEvent Event Type Type of Event, System or User Event Title Title ofEvent Event Data Data for the Event Event Time Time of the Event EventUser ID Foreign Key - To the user who generated the eventThe Event Journal may be uploaded to a PC with an ActiveSyncapplication.

Application Programmer Interface (API) for Third Party Developers

Some users may be developers who wish to develop extensions to theremote control application. Thus, the remote control application mayexpose functionality to third party developers to build extensions. Thesupport provided may be via custom configuration files that containthird party graphics and supplemental IR codes.

Exceptions

The remote control application running on a platform should exhibitgraceful behavior under exception condition. Some examples of suchconditions and appropriate responses for the remote control applicationare as follows:

Low power situation: A low power indication is generated by theunderlying PDA operating hardware while the remote control applicationhas focus. In this case the remote control application may issue anindication to the user and also take steps to minimize power consumptionby, for example, disallowing certain actions. Since the remote controlapplication should not lose any of setup and configuration informationwhen the PDA 1150 runs out of power, the remote control application maystore this type of data in non-volatile memory (if available) and mayinhibit any write activity (i.e., changes to that memory) during lowpower situations.

Low memory or low storage indication received: If the operating systemof the platform issues a low memory or low storage indication while theremote control application has focus, the remote control application mayissue an indication to the user, relinquish to the system any scratchmemory it has in use, inhibit setup or other activities which mayconsume additional storage, suspend journaling activity, and take anyother steps possible to minimize memory and storage requirements.

Installation of new (updated or extended) version of remote controlapplication: If the user installs a new version of the remote controlapplication this may detect that initial installation has already beenconfigured for a previous version of the application and may use thoseconfiguration settings as the default settings during any newinstallation/customization process that occurs.

Additionally, the remote control application may behave gracefully ifthe user temporarily changes the allocation of storage and programmemory while the remote control application is active. Installation of asupplemental version of the remote control application may gracefullyuse all of the configuration settings that currently exist.

Desktop Remote Control Software Applications

The desktop remote control applications 1104 are applications that areinstalled on the user's PC 1152 to support the remote controlapplication on the PDA 1150. Examples of PC remote control softwareapplications include:

Configuration Designer

The configuration designer 1134 may allow a user to create aconfiguration file on the PC that can be loaded into remote controlapplication.

Configuration Emulator

The configuration emulator 1136 may allow the user to review theconfigurations on the PC as if it were running on the target PDA.

Restore/Backup Configurations

This component 1130 may allow the user to backup and restoreconfiguration files and event journals over the ActiveSynccommunications conduit to/from the PC and the PDA 1150.

Update Checking/Supplemental Patch Download

This component 1132 may review current version numbers of remote controlapplication and IR code databases, verify them against a Web hostedupdate service, and inform the user whenever later versions areavailable for download. Optionally, this component may also perform theupdates automatically via the ActiveSync conduit.

Cultural and Political Requirements

The remote control application may be shipped with a world-wide IR codedatabase and may be used in many international locations. The remotecontrol application may thus support localization, for both language aswell as key layout variations to support regional settings. Withreference to FIGS. 23 a through 23 d, the room setup wizard inputrequirement at step 2 may be expanded from that shown in FIG. 16 c toinclude a drop-down list 230 from which various regions may be selected.Based on this input, the behavior of the remote control application maybe adjusted to more closely match the region in which it is being used.For example, the order in which IR device codes are presented to theuser for testing in the drop-down menu 1430 (FIG. 14 f) during devicesetup may be varied from region to region in order to place the mostlikely (i.e., most popular in that country or region) codes at the topof the list. Furthermore, even within the same IR code set,functionality and key assignments may also vary by region and may beautomatically adjusted based upon the region selected.

By way of further illustration, control page 4 of the same televisionbrand is shown in FIG. 23 c as it appears with the region set to “NorthAmerica,” and in FIG. 23 d as it appears with the region set to“Europe.” By way of background explanation, in Europe some televisionbroadcasts include a feature known as “Teletext” in which textualinformation such as news briefs, weather forecasts, etc. is encoded inthe vertical blanking interval of the television signal. Many Europeanmarket television sets thus may include the capability to decode anddisplay this information under user control. Accordingly, with theregion set to “Europe” the remote control application displays soft keysfor the functions required to cause control 232 and navigate 234 signalsto be transmitted to influence the Teletext data display. Further, ascan be seen by comparing keys 232 in FIG. 23 c versus 23 d,manufacturers may offer different features in different markets whichare invoked by similar control functions—in this instance “Picture inPicture” for the North American market versus Teletext for the Europeanmarket.

It will also be appreciated that this region selection parameter mayalso be used internally by the remote control application for otherpurposes as well, for example it might be used to select one of severalpossible sets of “Help” data in different languages, either pre-loadedin the PDA 1150 or obtained via download from the desktop PC 1152 or Webhosted services 1106. All string literals may be stored in the Windowsbrand OS resource file to provide a central location to performlocalization work.

While specific embodiments of the present invention have been describedin detail, it will be appreciated by those skilled in the art thatvarious modifications and alternatives to those details could bedeveloped in light of the overall teachings of the disclosure.Accordingly, the particular arrangement disclosed is meant to beillustrative only and not limiting as to the scope of the inventionwhich is to be given the full breadth of the appended claims and anyequivalents thereof.

1. A readable media embodied in a non-transient, physical memory devicehaving stored thereon instructions executable by a hand-held remotecontrol device for displaying information in a display of the hand-heldremote control device, wherein the instructions perform stepscomprising: displaying in the display of the hand-held remote controldevice a plurality of iconic representations each corresponding to arespective one or more of a plurality of controllable functions of atleast one home appliance which the hand-held device can be set up toremotely control; and receiving input for assigning to at least one ofthe plurality of iconic representations displayed in the display of thehand-held remote control device one or more command codes selected froma library of command codes stored in a memory of the hand-held remotecontrol device whereupon, in response to subsequent selection of the atleast one of the plurality of iconic representations when displayed inthe display of the hand-held remote control device, the hand-held remotecontrol device will use the one or more command codes selected from thelibrary of command codes stored in the memory of the hand-held remotecontrol device to issue a transmission to control the one or more of theplurality of controllable functions of the at least one home appliancecorresponding to the at least one of the plurality of icons, andwhereupon an appearance of the one of the plurality of iconicrepresentations displayed in the display of the hand-held device ischanged from a first appearance to a second appearance to therebyreflect that the at least one of the plurality of iconic representationshas been assigned one or more command codes selected from the library ofcommand codes stored in the memory of the hand-held remote controldevice.
 2. The readable media as recited in claim 1, wherein theinstructions provide for downloading at least a portion of the libraryof command codes into the memory of the hand-held remote control device.3. The readable media as recited in claim 1, wherein the plurality oficonic representations comprise a home theater page of the hand-heldremote control device.
 4. The readable media as recited in claim 1,wherein the plurality of iconic representations comprise a favoritechannels page of the hand-held remote control device.
 5. The readablemedia as recited in claim 1, wherein the plurality of iconicrepresentations comprises a device page of the hand-held remote controldevice.
 6. The readable media as recited in claim 1, wherein the secondappearance comprises an image and the first appearance comprises anoverlay that is provided to the image.
 7. The readable media as recitedin claim 1, wherein the display comprises a touch screen display.